FILTER MODE ACTIVE

#LLM inference

Records found: 5

#LLM inference20/11/2025

LLM Inference Showdown: vLLM vs TensorRT-LLM vs HF TGI v3 vs LMDeploy

'A concise technical comparison of vLLM, TensorRT-LLM, Hugging Face TGI v3 and LMDeploy, highlighting when to use each stack for production LLM inference based on throughput, latency and KV behavior.'

READ →

#LLM inference07/11/2025

Battle of the Runtimes: Top 6 LLM Inference Engines to Use in 2025

'A practical comparison of the top 6 LLM inference runtimes in 2025, highlighting design tradeoffs, KV cache strategies, performance profiles, and where each engine fits in production.'

READ →

#LLM inference06/10/2025

StreamTensor: Streaming LLM Intermediates from PyTorch to FPGA Dataflows

StreamTensor compiles PyTorch LLM graphs into stream-scheduled FPGA dataflows using an itensor type and LP-based FIFO sizing to reduce DRAM round-trips, lower latency, and improve energy efficiency.

READ →

#LLM inference14/07/2025

Fractional Reasoning: Enhancing LLM Inference with Adaptive Depth Control

Fractional Reasoning introduces a model-agnostic method to adaptively control reasoning depth in LLMs, enhancing performance and efficiency on complex reasoning tasks.

READ →

#LLM inference31/05/2025

Microsoft’s WINA: Revolutionizing Efficient Inference for Large Language Models Without Training

Microsoft and collaborators introduce WINA, a novel training-free sparse activation method that significantly improves efficiency and accuracy in large language model inference by leveraging both neuron activations and weight norms.

READ →